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What does the best evidence say about what works? 


• Bad news: There is relatively limited evidence to guide practitioners on ways to reduce gun 
violence or help individuals desist from crime 

• We’re not starting from scratch, but we will need to experiment 

• Good news: There are lots of things we could try! 

• This talk will give you the highlights from research that may be useful to consider when designing 
an intervention for New Orleans 

• Offers ideas + cautionary tales 




How will we know if our intervention is effective? 


• Measuring the effect of a program on any outcome requires knowing the counterfactual: 

• What would have happened in the absence of the program? 

• Selecting participants based on things like motivation or criminal history makes it 
impossible to tell whether a program had any effect 

• We need a control group — people who are similar to those in the treatment group, but 
who do not have access to the program 

• Randomizing access across individuals or groups is ideal 




How will we know if our program is cost-effective? 


• Once we have a control group, we can measure the program’s effects on a variety of outcomes 
by linking individuals to administrative data: 

• Future arrests, convictions, incarceration spells 

• 91 I calls, ER visits, injuries, deaths 

• Social service receipt (TANF, unemployment benefits) 

• Educational outcomes (high school graduation, college enrollment) 

• Employment, taxes paid 

• Effects on their kids (education, health, criminal justice involvement) 


• Program benefits = control group outcomes - treatment group outcomes 

• Example: Suppose the cost of incarceration is $ 100 per day, and we find that the average 
person in the treatment group spends 30 days in jail over the next 3 years, while the 
average person in the control group spends 40 days in jail during the same time period 

• The savings in terms of reduced incarceration would be 10 days/person*$ 100/day = 

$ 1,000 per person 


Summing up all of benefits allows us to estimate the total savings that the program produced 
• If those benefits exceed the cost of the program, then the program was cost-effective 





Examples of relevant research 


• Transitional job programs 

• Mental health treatment 

• Cognitive behavioral therapy 

• Multisystemic therapy 

• Substance abuse treatment 

• Medicaid expansion 

• Medication-assisted treatment 

• Wrap-around services 




Transitional jobs 


• What is it? 

• Provide temporary (e.g. 6 mos.) employment, to help individuals build soft skills and job- 
specific skills that might make them more appealing to employers 

• Usually at non-profits; goal is to find private sector employment after program ends 




Transitional Jobs Reentry Demonstration 


RCT: Effect of transitional employment in midwestern cities 

Valentine & Redcross (2015) 
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Quarter after Random Assignment (RA) 

No significant effect on recidivism. 




Center for Employment Opportunities evaluation 


RCT: Effect of transitional employment in New York 

Valentine & Redcross (2015) 
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Significant reduction in misdemeanor convictions. 







Transitional jobs upon reentry 


Effect of wrap-around services + transitional employment in Milwaukee 

Cook et al. (2015) 
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No significant effect on likelihood of re-incarceration. 









Transitional jobs 


• Punchline: 

• Transitional jobs do not have a long-term effect on employment, and in most cases do not 
reduce recidivism 

• Caveats: 

• If employment is valued for its own sake, then these programs demonstrate that the hard- 
to-employ will show up to work every day as long as the subsidy lasts 

• In this case, we could consider a jobs-of-last-resort program 

• Open questions: 

• Ongoing “enhanced” transitional job pilots are testing whether changing the model (e.g. 
adding CBT, placing people directly with private employers) can increase its effectiveness 




Addressing mental health needs 


• Mental illness, emotional trauma, and related issues are a problem for a large share of people in 
jail and prison 

• What if we invested more in addressing their needs? 


Mental health status of prisoners and jail inmates, by type 
of mental health indicator, 2011-2012 

Percent 



Source: Bureau of Justice Statistics, National Inmate Survey, 2011-2012. 


Prisoners and adult general population who met the 
threshold for serious psychological distress, 2009-2012 



Percent 

Source: Bureau of Justice Statistics, National Inmate Survey, 2011-2012; and 
Substance Abuse and Mental Health Services Administration, National 
Survey on Drug Use and Health (NSDUH), 2009-2012. 

























Cognitive Behavioral Therapy (CBT) 


• What is it? Form of psychotherapy that helps patients identify negative or inaccurate thinking 
so that patients can respond to challenges in a more effective way 

• Can be administered in individual or group settings 




Heller, et al. (2017) — CBT reduces reincarceration for juveniles 


• R.CT of Becoming a Man (BAM) in Cook County Juvenile Detention Center 

• BAM reduces 18-month readmissions by 0.7 per person — 32% of the baseline 

• Benefits in avoided social costs of crime are at least 5-times the cost of the program 



2 4 6 8 10 12 14 16 18 


Months since release 


With Day-of-Admit Fixed Effects 


Without Day-of-Admit Fixed Effects 

























Cognitive Behavioral Therapy (CBT) 


• Punchline: 

• In multiple, large U.S. studies, CBT has been very effective. But there are some places where 
the programs were not effective as implemented. 

• Open questions: 

• Is CBT more effective for some groups than others? 

• Will it be as effective when instructors have less training? (This will be necessary if we want 
to scale it.) 




Multisystemic Therapy (MST) 


• What is it? Mental health treatment that includes the family & communities of targeted youth 




Sawyer, et al. (201 I) — MST reduced criminal behavior in U.S. 


• R.CT in Missouri in 1980s, 176 court-involved youth (originally age 18-23) tracked for 22 years 

• Other studies found MST also reduced crime committed by the caregivers and siblings of the 
targeted youth 


Percentages and Odds of Rearrests and Civil Suits During 
Follow-Up by Therapy Condition 


Variable 

% 

OR 

95% Cl 



Criminal arrests 

Any felony 

IT 

MST 

Violent felony 

IT 

MST 

Nonviolent felony 

IT 

MST 

54.8 

34.8 

15.5 

4.3 

51.2 

34.8 

2.27 

4.08 

1.97 

[1.29, 4.01] 

[1.35, 12.36] 

[1.11,3.47] 

Any misdemeanor 


1.22 

[0.68,2.17] 

IT 

65.5 



MST 

60.9 



Civil suits 




Family instability 


2.08 

[1.17,3.47] 

IT 

47.6 



MST 

30.4 



Financial problems 


1.03 

[0.56, 1.88] 

IT 

31.0 



MST 

30.4 




Note. Sample sizes for therapy conditions are as follows: individual 
therapy (IT; n = 84); multisystemic therapy (MST; n = 92). OR = odds 
ratio; Cl = confidence interval. 









Fonagy, et al. (2018) — Recent, large RCT in England found no effect 


• Randomized families of I 1-17 year olds with moderate to severe antisocial behavior to receive 
MST or treatment as usual 

• Large study: 684 families in England 

• No significant differences in subsequent out-of-home placements or criminal convictions 



Effect of 

multisystemic therapy 

95% Cl p value 

Out-of-home placement 

1-25* 

0-77-205 0-37 

Time to first offence 

106t 

0-84-1-33 0-64 

*Odds ratio. fHazard ratio. 



Table 2: Logistic regression analysis of out-of-home placement and Cox 
proportional hazards model of time to first offence 









Multisystemic Therapy (MST) 


• Punchline: 

• Several RCTs have found that MST reduces subsequent criminal behavior of juveniles in the 
short and long run (relative to individual therapy). 

• Follow-up studies also show MST reduces criminal behavior but the siblings and caregivers of 
those initially randomized. 

• However, a recent, large RCT found no significant effects. 

• Caveats: 

• Several of these studies were conducted in Europe, where the criminal justice context (and 
treatment-as-usual received by the comparison group) are very different. 

• Sample sizes tend to be small. 

• Open questions: 

• For which groups is it cost-effective? 

• Will it still be effective if therapists have less training? 




Substance abuse treatment 


• Substance abuse is also a problem for a large share of people in jail and prison 

• What if we invested more in helping them manage their addictions? 

Inmates and adult general population who met the criteria 
for drug dependence or abuse, 2007-2009 

Percent ■ Abuse 

■ Dependence 
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Source: Bureau of Justice Statistics, National Inmate Surveys, 2007 and 2008-09; 
and Substance Abuse and Mental Health Services Administration, National Survey 
on Drug Use and Health, 2007-2009. 











Medicaid expansions increase access to treatment & reduce crime 


• Wen, Hockenberry, and Cummings (2017): 

• Medicaid increases access to substance 
abuse treatment 

• Medicaid expansions reduce crime, 

2001-2008 

• Effects on employment for this group? 

• See also Vogler (2017) for similar findings from 
ACA Medicaid expansions 
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Medication-assisted treatment 


• What is it? Providing medication (e.g. methadone, buprenorphine, naltrexone) to help 
individuals limit substance abuse 




Medication-assisted treatment 


• Rigorous evidence is very limited: 

• Lee et al. (2015) 

• Pilot RCT of extended-release naltrexone for opioid-dependent men, beginning I week 
before release from NYC jail 

• Comparison group: No medication 

• No significant effect on re-incarceration but N=34 

• Lobmaier et al. (2015) 

• RCT of naltrexone implant beginning I month before release 

• Comparison group: Methadone 

• 46 volunteers with histories of heroin use were randomized to each treatment 

• No significant difference in self-reported criminal activity across interventions 

• Gordon et al. (2017) 

• RCT comparing various combinations of buprenorphine and counseling, before and after 
release, for inmates with histories of opioid dependence 

• Follows participants for 12 months after release 

• No significant difference in self-reported criminal activity across interventions 





Medication-assisted treatment 


• Punchline: 

• This evidence is too thin to tell us much at all 


• Caveats: 

• Samples are very small 

• Studies typically don’t track administrative data on recidivism (indeed many other studies 
exist that only look at drug use) 

• Comparison group is often another form of treatment that could be equally effective 


Open questions: 

• Everything 




What if we combine lots of strategies into one? 


• Since the formerly-incarcerated have many needs, perhaps we need to address all of them at 
once in order to reduce recidivism 




Wrap-around services 


• What is it? 

• Multi-faceted programs that aim to address a variety of needs after release (e.g. housing, 
employment, substance abuse, CBT, case management) 

• Very labor-intensive, expensive intervention — but if it works it might be worth it 





Grommon, et al. (2013) — no effect on rearrest or reincarceration 


• R.CT of a highly-respected program prioritizing drug treatment, for medium- and high-risk male 
parolees 

• No significant effects on rearrest or reincarceration 

• The treatment group did worse on average 


TABLE 3 Summary of Relapse and Recidivism ANOVA Outcomes by Experimental Condition 
(N= 511) 


Outcome 

M 

SE 

F 

df 

< 

Relapse: At least one positive 






Treatment 

.75 

.03 

.64 

1, 509 

.001 

Control 

Relapse: Proportion positive 

.71 

.03 




Treatment 

.26 

.02 

4.46* 

1, 509 

.01 

Control 

.21 

.02 




Rearrest 






Treatment 

.30 

.03 

.74 

1, 509 

.001 

Control 

Reincarceration 

.27 

.03 




Treatment 

.36 

.03 

1.49 

1, 509 

.003 

Control 

.31 

•03 




Technical reincarceration 






Treatment 

.21 

.02 

1.56 

1, 509 

.003 

Control 

New sentence reincarceration 

.17 

.02 




Treatment 

.14 

.02 

.06 

1, 509 

.000 

Control 

.14 

.02 





*p< .05. 












Wrap-around services 


• Punchline: 

• The best evidence on this category of intervention suggests it is not effective 

• Caveats: 

• Some aspects of wrap-around services may be more cost-effective than others, but trying 
to do everything at once may make it impossible to do anything well 

• Open questions: 

• Are these programs ineffective because they’re tough to implement well? Or because high- 
intensity interventions act as a tether to the criminal justice system? 




Moving forward 


• Emphasis on employment might be misguided 

• A job may be nice to have, but not necessary or sufficient 

• Addressing other challenges first may make it easier to build a stable life that includes 
steady employment 

• But trying to address all needs at once may make it impossible to address any needs well 

• This is a really tough problem to solve 

• We need to get serious about rigorously evaluating what we try — even the programs that 
currently have some evidence behind them 

• We should assume that most things we try will fail 
• Be humble and aim to fail quickly 

• Keep trying until we figure out what works 





Thank you! 


Email 


• • 


jdoleac@tamu.edu 




